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ABSTRACT 

Traditional  maintenance  training  curricula  use  paper  technical  manuals  to  teach  ‘Iridcs  of  the  trade”  in 
corrective  and  preventive  maintenance  which  require  multiple  adjustments,  lengthy  calibrations  and  labor 
intensive  test/checkout  procedures.  Rotation  often  delays  personnel  assignment  to  a  platform  which  re¬ 
quires  these  maintenance  skills.  The  non-availability  of  skilled  maintenance  personnel  poses  a  tremen¬ 
dous  threat  to  readiness  conditions  for  mission  critical  equipment.  In  order  to  best  support  the  Navy’s 
maintenance  requirements  and  improve  maintenance  efficiency,  e  Naval  Research  Laboratory  at  Sten¬ 
nis  Space  Center,  Mississippi  developed  an  Expert  Maintenance  Advisor  (EMA). 

This  paper  discusses  an  approach  which  implements  Expert  Systems  technology  methods  to  develop 
an  expert  maintenance  reference.  It  also  addresses  the  impact  of  performance  support  systems  and  in¬ 
teractive  electronic  technical  manuals  in  maintenance  training. 

INTRODUCTION 


The  Expert  Maintenance  Advisor  (EMA)  is  an  expert  maintenance  training  reference  tool  and  perform¬ 
ance  support  system  for  system  level  components.  It  was  developed  to  augment  rather  than  remove  or 
replace  the  need  for  maintenance  personnel.  The  heart  of  the  system  is  an  intelligent  interactive  elec¬ 
tronic  technical  manual  which  is  not  just  an  automated  technical  manual  or  page  turner  but  an  expert  refer¬ 
ence  tool.  Its  current  configuration  uses  commercial-off-the-shelf  hardwar^software  to  achieve  faster 
access/data  retrieval  time  and  embedded  subject  matter  expertise  to  provide  increased  user  proficiency. 

Its  integration  into  maintenance  training  curricula  will  influence  maintenance  training  objectives  through 
rapid  diagnostics,  troubleshooting,  fault  localization  and  increased  maintenance  proficiency.  An  EMA  re¬ 
inforces  maintenance  training  objectives  because  on-line  expert  advice  is  a  part  of  the  system  from  cradle- 
to-grave  and  most  importantly  available,  during  training  at  the  schoolhouse,  ashore  and  afloat.  Figure  1 
shows  a  description  (Kearsley,  1982),  of  learning  efficiency  as  a  curvilinear  relationship  between  time  and 
proficiency.  The  novice  quickly  masters  basic  material  b^use  there  is  a  rapid  acquisition  of  new  knowl¬ 
edge  and  skills  at  the  beginning  of  instruction.  As  the  complexity  and  sophistication  of  the  task  increase, 
the  time  required  for  incremental  improvements  in  proficiency  lengthen,  thus  skill  degradation  is  rapid  with¬ 
out  application  reinforcement. 


Two  major  EMA  benefits  are  paper  cost  and  weight  reductions.  Presently,  within  the  Navy/Marine 
Corps,  237  million  drawings  and  15  million  technical  manuals  costs  $4,000,000,000.  The  USS 
Vincennes  stores  23.5  tons  of  paper  above  the  main  deck.  This  is  more  tonnage  of  paper  than  weap¬ 
ons,  therefore  paper  weight  is  a  BIG  deal.  Paper  cost/  weight  reductions,  subject  matter  expertise  avail¬ 
ability,  and  portability  benefits  alone  are  value-added  to  learning  efficiency,  performance  efficiency,  and 
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maintenance  effectiveness;  and  cost  savings  for  technical  information  reproduction  and  storage  require* 
ments.  Army  and  Air  Force  statistics  are  of  similar  magnitude. 

Expertise  can  be  defined  as  the  specialized  knowledge  that  permits  an  individual  to  excel  at  a  given 
task.  This  specialized  knowledge  is  typically  obtained  through  extensive  experience,  after  training  in  the 
basics  has  been  assimilated.  It  is  not  a  commodity  that  can  be  easily  or  quickly  transferred  from  one  indi¬ 
vidual  to  another.  It  can,  however,  be  readily  available  by  one  individual  to  another  through  expert  con¬ 
sultation.  The  primary  puqxrse  for  embedding  such  expertise  provides  expert  guidance  to  an  intelligent 
user  with  less  experience,  without  requiring  a  human  expert  to  be  present  The  techniques  for  capturing 
expert  knowledge  and  structuring  it  into  a  workable  confer  program  are  collectively  categorized  as  Ex¬ 
pert  Systems  development  (Holland,  1986). 

The  misdiagnosis  of  a  problem  can  put  a  combat  system  out  of  action  and  uselessly  deplete  the  sup¬ 
ply  of  the  component  which  is  removed  and  replaced.  Traditionally,  skill  in  troubleshooting  came  about 
as  a  by-product  of  years  of  experience  with  a  system,  and  an  apprenticeship  senred  under  a  master  trou¬ 
bleshooter.  This  critical  skill  became  even  more  valuable  because  of  the  costly  manner  in  which  it  was 
acquired.  These  costs  include  apprenticeship,  journeyman,  and  master  troubleshooting  skills  develop¬ 
ment  which  extend  over  the  military  and  post-military  careers  of  the  trainer  and  trainees.  Conventional 
classroom  training  requires  logistics,  travel,  courseware,  and  firmware  costs.  The  payoff  for  training  dol¬ 
lars  invested  comes  from  improved  job  performance.  The  best  evidence  of  this  was  ^monstrated  dur¬ 
ing  the  Persian  Gulf  War.  The  cost  of  maintenance,  measured  as  the  expense  of  restoring  operational 
readiness,  is  probably  determined  by  the  accuracy  of  fault  identification  more  than  any  other  factor.  De¬ 
termining  a  system  problem  doesn't  seem  costly,  after  all  nothing  is  being  consumed  except  time.  But 
the  fact  is  that  time  is  often  the  most  costly  resource.  Therefore  the  true  measure  of  valuable  training  is 
operational  readiness. 

The  challenge  we  must  meet  is  reduction  of  the  time  required  to  gain  this  experience.  In  this  sense, 
training  is  synthetic  experience,  and  as  with  most  synthetics,  there  are  aspects  which  are  effective  in  sub¬ 
stitute  form.  Even'more  desirable  is  an  increase  in  efficiency,  where  a  higher  level  of  skilled  perform¬ 
ance  is  reached,  even  though  costs  are  reduced.  Cost  effectiveness  is  a  baseline  requirement,  but  in 
the  foreseeable  future,  those  who  can  bring  about  cost  efficiency  will  be  meeting  the  true  need  (Bresee  & 
Greenlaw,  1 992).  Certainly,  money  spent  for  training  using  an  EMA  is  a  wise  investment  which  yields  a 
solid  investment  return,  viz.,  increased  maintenance  proficiency. 

SYSTEM  FEATURES 

Expert  Systems  development  techniques  capture  expert  knowledge  and  structure  it  into  a  workable 
compute^ogram  (Holland,  1986).  An  Interactive  Electronic  Technical  Manual  (lETM)  combines  artifi¬ 
cial  inteiligencb^^h  electronic  information  delivery  techniques  in  order  to  create  a  paperless  troubleshoot¬ 
ing  and  repair  emitonment,  (Long  &  Lontos,  1 992).  The  Expert  Maintenance  Advisor  couples  Expert 


Systems  technology  and  lETM  techniques  to  devetop  a  reference  tool  that  provides  assistance  during 
troubleshooting,  system  diagnostics  and  maintenance  tum-on  procedures.  It  uses  erT4)edded  subjed 
matter  expertise  to  guide  maintenance  personnel  through  a  logical  event  sequence.  A  reference  section 
of  helpful  hints  provides  "rules  of  thumb”  to  reinforce  teamed  maintenance  skilte.  The  menu-driven  sys¬ 
tem  resides  on  a  laptop  computer  in  a  DOS  4.01/C  Programming  Language  environment.  It  requires  a 
120MB  hard  drive  and  4MB  of  RAM  which  includes  SMB  of  extended  RAM  for  storage  and  quick  retrieval 
of  textual  data  and  graphics.  Hypertext  and  highlighting  features  are  included  for  all  technical  manual  ref 


erences.  Figure  2  sho\NS  thase  references  in  bold  on  the  screen  atKl  allows  the  user  to  go  directly  to 
tttem  using  ttte  Fl  and  TAB  function  keys. 

Know^gebase  iqxlates  can  be  done  without  source  code  recompilation  and  indeperKient  library  rou¬ 
tines  generation  for  the  source  code  and  rule  language  make  the  system  adaptable  to  multipie  correc- 
tive/preventive  maintenance  sy^em  application  developments.  Its  portabHity  makes  it  easy  to  use  in 
limited  work  spaces  and  its  availability  on  electronic  media  enables  maintenance  personnel  to  perform 
tasks  rapidly.  Figure  3  shovrs  the  AN/USH-32  Signal  Data  Recorder-Reproducer  Set.  its  Expert  Mainte- 


Figure  3.  ANAJSH-32,  Expert  Maintenance  Advisor 


nance  Advisor,  and  one  of  its  conventional  maintenar^  manuals  (Harris  &  Romalewski,  1994). 

When  a  complex  system  fails,  the  maintenance  goal  is  to  efficiently  get  it  fixed  and  back  on  line.  The 
application  of  expert  systems  techniques  to  the  development  of  maintenance  advisors  offers  the  greatest 
potential  for  significant  increases  in  maintenance  performance.  These  performance  measurements  can 
not  be  documented  until  system  deployment,  utilization,  and  evaluation  statistics  are  available.  Mainte¬ 
nance  advisors  make  maximum  use  of  basic  maintenance  training  and  existing  built-in  test  capability 
while  providing  near  instantaneous  access  to  pertinent  maintenance  information. 

EMAs  provide  an  on-board  training  opportunity  for  the  novice  who  has  not  performed  maintenance  ac¬ 
tions  or  trained  maintenance  personnel  in  need  of  an  equipment  maintenance  refresher.  EMA  system 
developers  and  subject  matter  experts  link  equipment  maintenance  learning  objectives  and  other  mainte¬ 
nance  requirements  in  the  EMA  development  phase  using  input  from  shipboard,  shore-based,  and  train¬ 
ing  community  users.  Pop-up  windows,  helpful  hints,  textual/graphical  references  enhancements  are 
used  to  capture  the  user's  interactive  response  during  EMA  sessions.  Another  useful  feature  is  the  ses¬ 
sion  save  which  allows  the  user  to  leave  the  session  and  return  to  the  session  at  the  last  process  step. 

In  the  session  replay  mode  troubleshooting  problem-solving  routines  which  are  performed  by  the  user  can 
be  stored  and  recalled  later.  This  allows  users  to  receive  continuous  training  no  matter  where  they  are 


(Buehler,  1 992)!  Specific  maintenance  actions,  troubleshooting  sMHs  and  performance  are  effective 
when  the  cause  of  the  fault  and  its  relationship  to  system  components  are  visible  to  the  user. 

Expert  Systems  are  adaptable  in  shore-based  environments,  where  a  significant  portion  of  the  training 
pipeline  is  dedicated  to  student  familiarization  of  the  organization  and  use  of  the  technical  manual  con- 
terrts.  While  these  requirements  are  pertinent  to  effect  training,  the  greatest  irT^)act  is  seen  in  the 
length  of  training  courses  and  their  associated  costs.  Expert  Systems  provide  a  reduction  in  training  cost 
and  training  tirrre  without  degradation  of  training  effectiveness  in  a  shore-based  environment  as  well  as  a 
shipboard  environment. 

PERFORMANCE  SUPPORT  SYSTEMS  IN  MAINTENANCE 

Performance  Support  Systems  (PSS)  are  computer-based  systems  that  use  knowledge-based  sys¬ 
tems,  hypertext,  on-line  reference,  extensive  data  bases,  and  allied  technologies  to  provide  support  to  per¬ 
formers  on  the  job,  where  they  need  H,  when  they  need  H,  in  the  form  rTX>st  useful  to  them.  The  four 
basic  PSS  functions  are  librarian,  advisor,  instructor,  or  dofer  that  does  work  with  or  without  assistance 
from  the  human  performer  (Carr,  1 992).  They  are  an  asset  to  maintenance  technicians  because  they 
provide  a  specific  solution  to  an  identified  maintenance  problem.  Portability  and  ease  of  use  take  the 
strangeness  out  of  the  Expert  Systems  and  make  them  rTK>re  accessible  and  exciting  for  users.  They  en¬ 
hance  prior  training  knowledge  and  increase  skill  level  in  a  shore-based  as  well  as  in  a  shipboard  environ¬ 
ment.  It  has  been  proven  many  times  that  “if  you  don’t  use  it,  you’ll  lose  if.  Hov  <ever,  with  the  onset  of 
Expert  System  technology,  a  counter  theory  may  support  the  idea  that  If  you  don’t  use  it.  Expert  Systems 
can  revive  if.  When  a  complex  system  fails,  the  maintenance  goal  is  to  efficiently  get  it  fixed  and  back 
on  line.  Expert  Systems  technology  developments  offer  the  greatest  potential  for  significant  increases  in 
maintenance  proficiency.  EMAs  make  maximum  use  of  basic  maintenance  training  skills  and  existing 
built-in  test  capabilities  while  providing  near  instantaneous  access  to  pertinent  maintenance  information. 

An  EMA  is  a  portable  computer,  programmed  using  Expert  Systems  techniques  to  give  a  mainte¬ 
nance  technician  expert  knowledge  on  a  specific  item  of  equipment.  The  technician  must  be  trained  only 
in  the  basics  of  troubleshooting  and  repair.  S/He  does  not  need  specialized  training  on  the  unit  requiring 
repair.  The  EMA  itself  contains  the  expert  information  acquired  from  system  developers,  maintenance 
manuals^and  subject  matter  experts.  This  broad  spectrum  of  information  resides  within  the  maintenance 
advisor  and  is  readily  available  to  maintenance  personnel.  It  does  not  require  a  paper  technical  manual 
search  to  retrieve  the  necessary  information  to  continue  with  maintenance  procedures.  As  shown  in  Rg- 
ure  2,  the  screen  highlights  reference  links  (text  and  graphics)  to  pertinent  information  for  all  maintenance 
procedures.  The  availability  of  the  complete  text/graphics  information  from  the  technical  manual  and  em¬ 
bedded  expert  knowledge  in  the  form  of  helpful  hints  are  key  features  of  the  EMA.  Access  to  this  infor¬ 
mation  allows  less  experienced  maintenance  technicians  to  perform  with  on-line  expertise.  The  EMA 
functions  as  a  supprort  system  to  keep  systems  operational,  I.e.,  reduce  meantime  between  failure  and  im¬ 
prove  maintenance  efficiency. 

PSS  offer  a  fertile  field  for  knowledge-based  technology.  They  represent  a  solution  looking  for  a  prob¬ 
lem  by  identifying  hOw  performance  can  be  improved  and  using  the  technology  that  best  accomplishes 
this  goal  effectively  and  efficiently.  Support  to  perfonoers  on  the  job  where  they  need  it,  when  they  need 
it,  and  in  form  most  useful  to  them  is  critical  in  system  maintenance.  The  EMA  functions  as  an  advisor 
which  embodies  and  shares  some  specialized  expertise  necessary  to  carry  out  the  maintenance  task. 

An  added  feature  of  a  properly  designed  performance  support  system  not  only  supports  performance  but 
develops  the  performers  skills  (Carr,  1992). 

The  integration  of  interactive  electronic  technical  manuals  and  Expert  Systems  technology  advances 
state-of-the-art  equipment  maintenance  techniques.  Conventional  training  coupled  with  these  techniques 
increases  trainee  and  training  objective  effectiveness.  The  advent  of  a  paperless  Navy  and  Computer- 
Aided  Acquisition  Logistics  Support  (CALS)  initiatives  make  this  Expert  Systems  technology  approach  a 
bellwether  in  maintenance  proficiency. 


Expert  Systeins  techrralo^  integrates  lETM  into  training  requirements  arxl  upgrades  training  capabili¬ 
ties.  It  also  supports  CALS  initiatives  to  improve  systems  quality,  reduce  development  time,  and  avoid 
cost  through  use  of  computer  technology.  This  approach  puts  the  user  in  the  driver’s  seat  by  placing  the 
primary  focus  on  satisfying  end  user  requirements.  It  supports  the  “create  once  use  many”  philosophy 
and  benefits  both  technical  information  and  end-user  computer  systems  applications. 

EMA  DEVELOPMENT  AND  USER  EVALUATION 

The  EMA  was  developed  using  C  Programming  Language  and  a  backward  chaining  inference  engine 
to  manipulate  facts,  rules,  and  goals.  Technical  manual  logic  diagrams,  preliminary  procedures/general 
notes,  and  the  maintenance  tum-on  procedure  are  the  rule-base  foundation.  Knowledge  acquisition  ses¬ 
sions  were  conducted  with  subject  matter  experts  to  embed  Maintenance  Hints  for  the  system. 

The  EMA  was  evaluated  by  trainees  and  a  private  contractor  during  its  operational  suitability  testing. 
The  evaluation  summary  is  listed  below. 

1)  Easy  to  use 

2)  Accurate 

3)  An  Asset  To  On-Going  Automated  Efforts 

4)  Provides  reliabiiity/supportability 

5)  Increases  corrective/preventive  troubleshooting  efficiency 

These  are  preliminary  evaluation  results.  A  comprehensive  EMA  evaluation  can  best  be  determined 
after  it  has  been  used  extensively  in  the  problem  domain.  The  final  evaluation  will  answer  questions 
which  address  decision  making,  control  strategy,  inference  nile  correctness/consistence/completeness, 
conclusion  organization/order,  and  level  of  detail  capabilities.  Users  in  the  field  demand  more  that  just 
high-  quality  performance;  they  want  a  system  to  be  fast,  reliable,  easy  to  use,  easy  to  understand,  and 
very  forgiving  when  they  make  mistakes.  Thus  the  EMA  needs  extensive  field  testing  before  it  will  be 
ready  for  Fleet  use  (Waterman,  1986). 

EMA  CURRENT  USE 

The  EMA  is  not  currently  being  used  in  fleet  maintenance.  Its  next  testing  phase  is  validation  and  veri¬ 
fication.  The  nature  of  Integrated  Logistics  Support  for  system  deployment  to  training  facilities,  organiza¬ 
tional/  interme  liate/depot  level  maintenance  facilities,  or  shipboard  requires  comprehensive  planning  and 
scheduling.  T  i-aining  and  shore-based  maintenance  facilities  will  probably  be  the  first  extensive  system 
users. 

EMA  LIMITATIONS  AND  FUTURE  ENHANCEMENTS 

Limitations  cited  in  the  evaluations  were  graphics  viewing  resolution,  graphics  speed,  and  visual  volt¬ 
age  level  readout  observations.  Graphics  viewing  resolution  and  speed  can  be  increased  using  newly 
developed  graphics  packages.  System  acquisition  and  display  of  digital  readout  voltage  levels  using  ana- 
log-to-digital  electronic  interface  between  the  EMA  and  AN/USH-32  are  feasible.  Computer  and  elec¬ 
tronic  firmware,  hardware,  and  software  are  available  to  resolve  these  limitations  and  they  will  be 
incorporated  as  system  modifications.  Rapid  advancements  in  these  product  areas  will  keep  Expert  Sys¬ 
tems  technology  developments  for  equipment  maintenance  on  the  leading  edge. 


CONCLUSION 


Expert  Maintenance  Advisors,  Performance  Support  Systems  arxl  Interactive  Eiectrtmic  Technical 
Manuals  impact  equipment  maintenance  requirements  significantly  in  the  wake  of  cutbacks  and  rightsiz¬ 
ing.  They  are  the  answer  to  cost  effectiveness  without  lost  in  performance  efficiency.  Expert  Systems 
technology  lends  itself  to  cost  effective  implementation  using  commercial-off-the-shelf,  government-off-the- 
shelf  and  non-developmental  items.  Performance  is  the  measure  of  success  in  any  operational  system, 
therefore  readiness  is  the  key  to  performance  efficiency  in  maintenance.  System  level  component  main¬ 
tenance  relies  heavily  on  the  training  community's  capability  to  provide  personnel  with  the  necessary  skills 
to  keep  systems  in  a  fully  operational  condition.  Ex^rt  Systems  technology  offers  the  best  approach  to 
achieving  maintenance  proficiency  by  initiating  new  maintenance  curricula  and  reinforcing  learned  mainte¬ 
nance  training  skills. 
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